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5.1.3 Conditioning and Independence 

We have discussed conditional probability before, and you have already seen some problems 
regarding random variables and conditional probability. Here, we will discuss conditioning for 
random variables more in detail and introduce the conditional PMF, conditional CDF, and 
conditional expectation. We would like to emphasize that there is only one main formula 
regarding conditional probability which is 

P(A n B) 

P(A\B) = — , when P(B) > 0. 

Any other formula regarding conditional probability can be derived from the above formula. 
Specifically, if you have two random variables X and Y, you can write 

P(X E C,Y ED) 

P(XE C\Y E D) = - -, where C, D c R. 

Conditional PMF and CDF: 

Remember that the PMF is by definition a probability measure, i.e., it is P(X = x /( ). Thus, we 

can talk about the conditional PMF. Specifically, the conditional PMF of X given event A, is 
defined as 


Px\A(x,) = P(X=Xi\A) 

P(X = x - t and A) 
= P(A) 


Example 5.3 

I roll a fair die. LetXbe the observed number. Find the conditional PMF ofXgiven that we 
know the observed number was less than 5. 


• Solution 

o Here, we condition on the event A = {X < 5}, where P(A) 


4 

6- Thus, 
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P x]A (\) = P(X= 1|X<5) 

P{X = 1 and X< 5) 
P(X< 5) 

P(X = 1) 1 

~~ P{X< 5) ~~ 4' 

Similarly, we have 

p x\A@) = Px\aQ) = P x\aW = 4 ’ 

Also, 

p x\A( 5) = ^xm(6) = 0. 



Conditional PMF of X Given Y: 

In some problems, we have observed the value of a random variable Y, and we need to update 
the PMF of another random variable X whose value has not yet been observed. In these 
problems, we use the conditional PMF of X given Y. The conditional PMF of X given Y is 
defined as 
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p x\ ) Px i I yp = p ( x=x i \ Y = yp 

P(X = Xp Y = }' ■) 
P(Y = yp 
p xyi x v yp 

p ftP ' 


Similarly, we can define the conditional probability of Y given X: 

P Y\X^j I x i> = P(Y = yj \X= x.) 

p xP x pYp 

p x( x i) 


For discrete random variables X and Y, the conditional PMFs of X given Y 
and vice versa are defined as 


p x\P x i\yp 


p r\x(yp x i ) 


p xP x pyp 

P &P 

p x)^ x o yp 
p x( x i) 


for any x i G R x andyy G R y . 


Independent Random Variables: 

We have defined independent random variables previously. Now that we have seen joint PMFs 
and CDFs, we can restate the independence definition. 


Two discrete random variables X and Y are independent if 
Pxyix, y ) = P x (x)P jfv), for all x, y. 
Equivalently, X and Y are independent if 

Fxyix, y) = F x (x)F } .{v), for all x, y. 
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p x\ P x i I yp = P{X=x i \ Y= yp 

p xP x pyp 

p pyp 

p x( x i) p Pyp 

p pyp 

= p P x P 

As we expect, for independent random variables, the conditional PMF is equal to the marginal 
PMF. In other words, knowing the value of Y does not provide any information about X. 

Example 5.4 

Consider the set of points in the grid shown in Figure 5.4. These are the points in set G defined 
as 


G = {(x,y)\x,y E Z, \x \ + \y \ < 2}. 

Suppose that we pick a point ( X, , Y) from this grid completely at random. Thus, each point has 
1 

a probability of — of being chosen. 

a. Find the joint and marginal PMFs ofXand Y. 

b. Find the conditional PMF of X given 7=1. 

c. Are X and 7 independent? 

• Solution 

o a. Here, note that 

r xy = g = {(*,7)1 x ,y e Z, |x| + \y\ < 2}. 

Thus, the joint PMF is given by 

(-77 ) ^ G 
otherwise 

To find the marginal PMF of A, P x (i), we use Equation 5.1. Thus, 


p xP X ’ 7) 


1 

13 
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P^(-2) = J P xy (-2,0)=-, 

3 

p ^- l ) = p xyt-^ -1)+ ^-1,0)+^-1,1)= 

^0) = Pxyi 0, - 2) + PxA 0, - 1) + P^O, 0) 

5 

+ PxA®> 1 ) + p xy$> 2 ) = ~, 

3 

Px(\) = PxA^’ - 1) + ^_xy(l, 0) + Pj^y(l, _ 1) = ’ 

P^(2) = P^<2,0)=-. 

Similarly, we can find 


^y(/) 


13 

for 7 = 2 , 

-2 

3 

13 

for J = - 

1,1 

5 

13 

o> 

II 

o 


0 

otherwise 



We can write this in a more compact form as 


P^k) = Py(k) = 


5 — 21 it I 


13 


for A: = -2, - 1,0, 1,2. 


b. For i = - 1, 0, 1, we can write 


p x | y( ? 'l 


P xAp 1 ) 

PA i) 


13 1 

X = 3' 
13 


for i = - 1, 0, 1. 


Thus, we conclude 
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Px\rf \') 



for i = -1,0,1 
otherwise 


By looking at the above conditional PMF, we conclude that, given 7 = 1, X 
is uniformly distributed over the set { - 1, 0, 1}. 
c. X and Y are not independent. We can see this as the conditional PMF of X 
given 7=1 (calculated above) is not the same as marginal PMF of X, P x (x). 


Conditional Expectation: 

Given that we know event A has occurred, we can compute the conditional expectation of a 
random variable X, E[X\A~\. Conditional expectation is similar to ordinary expectation. The 
only difference is that we replace the PMF by the conditional PMF. Specifically, we have 

E[X\A-\ - 

x i eR X 

Similarly, given that we have observed the value of random variable 7, we can compute the 
conditional expectation of X. Specifically, the conditional expectation of X given that 7 = y is 


E[X | 7 = y] = Yj x , P X\ Y< x i I y)- 
x i eR x 



Example 5.5 Let X and 7be the same as in Example 5.4. 
a. Find E[X\ 7 = 1 |. 
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b. Find E\X\ - 1 < Y< 2], 

c. Find E[ \ X\ \ - 1 < Y< 2], 

• Solution 

o a. To find E[X\ Y = 1], we have 

E[X\Y=l]= A-l')- 

Xi ER x 

We found in Example 5.4 that given 7 = 1, Xis uniformly distributed over 
the set { - 1, 0, 1}. Thus, we conclude that 

1 

E[X\Y= 1] = “( - 1 + 0+ 1) = 0. 

b. To find E[X\ - 1 < Y < 2], let A be the event that -1 < Y < 2, i.e., 

YE {0, 1}. To find E[X \A], we need to find the conditional PMF, P x ^ A (k), 

for k = - 2, 1, 0, 1, 2. First, note that 

Thus, for k = - 2, 1, 0, 1, 2, we have 

13 

P X \A&) = k > A )- 

So, we can write 
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P X \a(~ 2) = ~P{X= -2, A) 

13 1 

= -/^-2,0)=- 


W-1)=T^= -M) 


- 8 l/xy( l>°) + jP xy( 1 ’ !)] g _ 4 ’ 


p x\A 0)= -P(X=0,A) 


y[P xy (0,0) + P^(0,l)] = ^ = ^ 


p xA l)=~P(X=\,A) 


g Y p xyA, °) + p xyA-. 1)] 8 4 > 


^( 2 )= ~P{X=2,A) 


jPjd 2 ’ °) = g • 


Thus, we have 


£[Z|/I]-X 


XiER x 


1111 


= (- 2)- + ( - D- + (0)- + (1)- + (2)- = 0. 

c. To find E[ |X| | -1 < Y < 2], we use the conditional PMF and LOTUS. We 
have 


£[XM1 = Z 


x i G R X 


1 1111 
21 • - + I - 1 I • 7 + 0 • 7 + 1 • 7 +2 • - = 1. 

1 8 1 1 4 4 4 8 


Conditional expectation has some interesting properties that are used commonly in practice. 
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Thus, we will revisit conditional expectation in Section 5.1.5 . where we discuss properties of 
conditional expectation, conditional variance, and their applications. 

Law of Total Probability: 

Remember the law of total probability: If B^, f? 0 , f? 3 ,... is a partition of the sample space S, 
then for any event A we have 

P(A) = n B t ) = 

i i 

If 7is a discrete random variable with range R y = {y 1 ,y 2 , ■ ■ ■ }, then the events {Y = y x ), 

{Y = y 2 j, {Y = y 3 }, ••• form a partition of the sample space. Thus, we can use the law of total 

probability. In fact we have already used the law of total probability to find the marginal 
PMFs: 


p x( x )= yp = Z p x\ y( x I yjPJyp- 

VjERy VjeRy 

We can write this more generally as 

P(X G A) = ^ P(X G A | Y = Vj)Py(yj), for any set A. 

yj ER Y 

We can write a similar formula for expectation as well. Indeed, if B^B 0 , i? 3 ,... is a partition 
of the sample space S, then 


EX= Z^l BJPiBj). 
i 

To see this, just write the definition of E[X\ Z? ; ] and apply the law of total probability. The 
above equation is sometimes called the law of total expectation [2]. 


Law of Total Probability: 

P(X G A) = ^ P(X G A | Y = Vj)P yfvy), for any set A. 

VjERy 

Law of Total Expectation: 

1. If B 3 ,... is a partition of the sample space S, 
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EX= ^ j E[X\B j ]P(B j ) (5.3) 

i 

2. For a random variable X and a discrete random variable Y, 

EX= Y J E[X\Y = y j \P y tVj) (5.4) 

y, eR Y 


Example 5.6 

Let A ~ Geometric(p). Find EX by conditioning on the result of the first "coin toss." 

• Solution 

o Remember that the random experiment behind Geometric(p) is that we have a coin 
with P{H) = p. We toss the coin repeatedly until we observe the first heads. X is 
the total number of coin tosses. Now, there are two possible outcomes for the first 
coin toss: Ho r T. Thus, we can use the law of total expectation (Equation 5.3): 


EX = E[X | H]P(H) + E[X | T\P{T) 

= pE[X\H] + (\ - p)E[X\ 7] 

= p • 1 + (1 - p)(EX + 1). 

In this equation, E[X\ T] = 1 + EX, because the tosses are independent, so if the 
first toss is tails, it is like starting over on the second toss. Solving for EX, we 
obtain 


1 

EX = -. 
P 


Example 5.7 

Suppose that the number of customers visiting a fast food restaurant in a given day is 
N ~ PoissoniX). Assume that each customer purchases a drink with probability p, 
independently from other customers and independently from the value of N. Let Abe the 
number of customers who purchase drinks. Find EX. 

• Solution 

o By the above information, we conclude that given N = n, then A is a sum of n 
independent BernoullUp) random variables. Thus, given N = n, Ahas a binomial 
distribution with parameters n and p. We write 

X\N = n ~ Binomial{n,p). 
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That is, 

P A-|#I") = (^]P*( 1 -/>)”"*• 

Thus, we conclude 

E\X\N = n] = np. 

Thus, using the law of total probability, we have 

GO 

E[X\=Y J E[X\N=n]P N {n) 

n = 0 

00 

= X n P p d n ) 

n = 0 

00 

= />X nP i\( n ) = pE[N ] = pX. 

n = 0 


<— previous 
next —■> 
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